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"U. A human motion following controller for augmenting motion of items shown 
on^a computer display, the display being coupled to a computer of the type which 
consols positioning of the items through operating system controls, comprising: 



5 a camera for capturing frames of data corresponding to a first image of at least part 
of a user at th^^computer display; 



signal processing means coupled to the camera for (a) detecting differences between 
successive frames ofNiata corresponding to motion of the first image, and (b) 
10 communicating differences information to the computer to reposition display of the 
items through operating sysfkm controls, the items being repositioned on the display 
by an amount corresponding to the motion of first image. 



2. A controller of claim 1, wherein\the items comprise a computer cursor. 

3. A controller of claim 1, wherein the itfems comprise a scene view. 

4. A controller of claim 1, further comprising\a PC card for installation within 
the computer and for communication on a computer bus, the signal processing 

20 means being substantially resident with the PC card forxcommunicating differences 
information to the bus. 




5. A controller of claim 1, wherein the camera comprises means for capturing 
augmented frames of data corresponding to a second image of part Q|f the user at the 

25 computer display, the signal processing means further comprising means for 
detecting differences between successive augmented frames of data corresponding 
to motion of the second image and for communicating augmented diference 
information to the computer to reposition display of the items through operc^ing 
system controls, the items being repositioned on the display by an amouht 

30 corresponding to motion of the first and second images. 
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A controller of claim 1, further comprising frame difference electronics for 
stoihlng and subsequently subtracting pixel-by-pixel difference data. 

7. ANzontroller of claim 6, wherein the difference electronics comprise multiple 
5 frame meinory, a subtraction circuit, and a state machine controller/ memory 

addresser to control data flow. 

8. A controller W claim 1, further comprising N frame video memory for storing 
frames of image data. 

10 

9. A controller of claiiri 1, further comprising a DSP for implementing select 
algorithms on difference frame^^or^aw frames of image data. 

10. A controller of claim 9, further comprising memory selected from the group 
15 of EPROM and RAM. 

11. A controller of claim 9, further comprising means for interfacing the DSP to a 
PCI bus in the computer. 

20 12. A controller of claim 1, further comprising MPEG compression electronics for 
compressing video for the computer. 



13. A controller of claim 1, wherein the signal processing^means comprises frame 
differencing means for removing unchanged information from image frames. 



25 



14. A controller of claim 1, wherein the signal processing means comprises frame 
memory to buffer one or more image frames. 

15. A controller of claim 14, wherein the signal processing means comprises a 
30 frame differencer for reading a delayed frame from the frame memory and^for 

subtracting the delayed frame from a current image frame. 
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\6. A controller of claim 1, wherein the signal processing means comprises 
cori;elation means for determining row and column shifts corresponding to 
differences between a current image frame and a delayed image frame. 



5 17. A controller of claim 16, wherein the signal processing means comprises best 
fit algorithmmeans for minimizing the shiftis to provide alignment. 



18. A conti'oUervof claim 17, wherein the best fit algorithm means utilizes a peak 
detect algorithm. 



10 



19. A controller of claim 1, wherein the signal processing means comprises video 
cursor control for enabling and alternatively disabling cursor control. 



20. A controller of claim 19, wherein the video cursor control comprises means 
15 responsive to keystrokes at the computer. 



21. A controller of claim 19, wherein the video cursor control comprises means 
responsive to a blink of an eye of the user. 

20 22. A controller of claim 19, wherein the video^^ursor control comprises means 
responsive to sound generated by the user. 

23. A controller of claim 22, further comprising a microp1\pne to detect the sound. 

25 24. A controller of claim 1, wherein the signal processing means comprises a 
complex multiplier for providing a two dimensional inverse FFT operation. 

25. A controller of claim 24, wherein the signal processing means comprises a 
peak detect for determining a shift associated with aligning difference image 

30 
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A controller of claim 1, wherein the signal processing means comprises FFT 
means for providing a two dimensional FFT of image data. 



27. A controller of claim 1, wherein the signal processing means comprises means 
for identifying parts of the user, the parts being selected from the group of a hand, 
elbow, head^'n'eck, ears, and forehead. 



28. A controller of claim 1, wherein the signal processing means comprises means 
for detecting left and right movement of a head of the user and for shifting the items 
10 in response to the left and right movement. 



29. A controller of claim lAwherein the signal processing means comprises means 
for detecting rotational movemei^of a head of the user and for rotating the items in 
response to the left and right movement. 



15 



29. A controller of claim 1, wherein tne^ignal processing means comprises means 
for repositioning the items, if appropriate, at^pproximately every 1/30^^ of a second. 



30. A controller of claim 1, wherein the signal processing means comprises means 
20 for repositioning the items at a selected magnification as compared to actual 
movement of the user. 



31- A controller of claim 1, wherein the signal processing means comprises means 
for storing image data of a head of the user at various orientations relative to the 
25 camera and for correlating image data to the stored image data to define head 
orientation, the head orientation being used to reposition the items, 

32. A controller of claim 1, wherein the signal processing means comprises IR 
means for detecting heat associated with the user and for repositioning th^ items at a 
30 rate correlated to the heat. 
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^33. A controller of claim 32, wherein the items correspond to computer gaming 
display images. 

34. ^controller of claim 32, wherein the heat corresponds to user stress. 



35. A controller of claim 1, further comprising at least one other camera arranged 
to take images of at least a second part of the user. 



36. A controller of claim 35, wherein the one other camera takes image data in a 
10 second el ectromagneticx spectrum. 



37. A controller of claim l^^^^erein the camera comprises a DSP. 

38. A controller of claim 37, wherein the DSP processes difference information for 
15 the computer. 

39. A controller of claim 1, wherein the signal processing means comprises a CPU 
within the computer. 

20 40. A controller of claim 1, wherein the signaF^rocessing means comprises means 
for processing multiple image zones in frames of image data and for repositioning 
the items according to characteristics between zones. 

41. A controller of claim 40, wherein one zone \comprises image data 
25 corresponding to at least one eye of the user. 

42. A controller of claim 41, wherein the signal processing "kieans comprises 
means for determining a blink of the eye. 

30 43. A controller of claim 42, wherein the signal processing means "Comprises 
means for disabling and alternatively enabling cursor control based upon the Vl ink. 
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^4. A controller of claim 1, wherein the signal processing means comprises means 
for f^cessing image data to determine motion of at least one eye of the user and for 
repositioning the items based upon the motion. 



5 45. A controller of claim 1, wherein the signal processing means comprises means 
for processingVimage data to determine motion of a pupil of at least one eye of the 
user and for repol^tioning the items based upon the motion. 

46. A controller of oMm 1, wherein the camera comprises a zoom attachment for 
10 automatically zooming into a desired magnification of at least one eye of the user. 

47. A controller of claim \ wherein the camera comprises zoom means for 
automatically focusing on the user\as the user moves in distance from the camera. 

15 48. A controller of claim 47, wheiWn the signal processing means comprises 
means for enlarging or shrinking the item\pn the display in response to focusing by 
the zoom means. \ 

49. A controller of claim 1, wherein the signal p^cessing means comprises means 
20 for determining edges of a head of the user and ibr repositioning the items in 

response to movements of the edges. \ 

50. A controller of claim 1, wherein the signal processing means comprises means 
for isolating one or more objects held by the user and for repositioning the items in 

25 response to movement of the objects. \ 

51. A controller of claim 1, wherein the signal processing means compidses means 
for isolating one or more parts of the user and for repositioning the items in response 
to movement of the parts. \ 

30 ^ 
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A controller of claim 1, wherein the parts comprise at least one of a hand, 
head, and a foot. 




controller of claim 1, wherein the signal processing means comprises means 
for isolating one or more symbols associated with the user and for repositioning the 
items in response to movement of the symbols. 



54. A controlleivof claim 1^ further comprising a second camera constructed and 
arranged for viewing the user from above, the signal processing means having 
means for repositionin'gM:he items in response to movement detected from images in 
the second camera. 



55. A controller of claim 54, wherein signal processing means comprises means 
for repositioning the items in response to forward and backward movement of the 
user as detected by the second camera.^ 



56. A controller of claim 1, further comprising re-calibration means connected 
with the signal processing means for repositioning the items to an original position 
in response to a re-calibration event, 

57. A controller of claim 1, wherein the re-cali^ation means comprises a 
microphone and the event comprises a sound generated by\he user. 

58. A controller of claim 1, wherein the camera comprised the re-calibration 
means for detecting a blink of the user. 

59. A system for controlling a computer, comprising: 
a transducer for converting optical signals to electrical signals; 
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electronic means for converting electronic signals to digital data; 



signaKprocessor means for detecting motion in the digital data and providing a 
digital representation of said motion; 



communication means for entering one or more of the electronic signals, digital data, 
5 and digital representation into the computer to manipulate a computer display in 
response to the motion. 



if^i 60. A system of claim 59, wherein the computer comprises the signal processor 

means. 



61. A system of claim 59, wherein said signal processor comprises a digital signal 
i!|\lO processor separate from a CPU within the computer. 




62. A system of claim 59, wherein the tr'^nsducer, electronic means and signal 
processor are constructed and arranged into avsingle device in communication with 
the computer. 

63. A system of claim 59, wherein the communicatibn means comprises one of 
15 RSI 70 video, a PCI bus interface, a digital computer interface, a serial computer 

interface. 

64. A system of claim 59, further comprising means for reposittoning a computer 
cursor in response to the motion. 

65- A system of claim 59, wherein the transducer comprises one or morXx^f a 
20 visible CCD camera and an IR camera. 
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^6. A system of claim 59, wherein the transducer comprises a CCD camera 
having at least 2x2 imaging pixels. 



67. A system of claim 66, wherein the camera comprises optics with various fields 
of view. 



68. A system ofclaim 59, wherein the transducer comprises one of a CCD or a 
CMOS integrated circuit with digital outputs. 



69. A system of claim 68^^herein the transducer generates RSI 70 ouput. 

70. A system of claim 68, wherein the transducer generates RSI 70 digital output. 

71. A system of claim 68, wherein\the transducer generates digital resolutions of 4 
bits or greater 

72. A system of claim 59, wherein the signal processor comprises a video frame 
memory. 

73. A system of claim 59, wherein the signal processor comprises frame 
difference functionality. 

74. A system of claim 59, wherein the signal processor coiiiprises video frame 
difference memory. 

75. A system of claim 59, wherein the signal processor comprises correlation 
functionality. 

76. A system of claim 59, wherein the signal processor comprises means J 
determining best fit motion. 
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A system of claim 59, further comprising means for controlling cursor 
movehient. 

78. A system ot^iaim 59, further comprising means for segmenting video images 
to provide multiple di^tel representations of the motion corresponding to different 
portions of the digital repre^ntation. 

79. A system of claim 78, wherein thbvQptical signals are generated through image 
acquisition of a portion of a human. 

80. A system of claim 78, wherein the optical signals ai-^eenerated by viewing 
multiple features of a human. 

81. A system of claim 59, further comprising neural net means for learmM user 
motion over time. 
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